Cache memories can improve the overall performance of a computing device by providing improved access times to larger, slower memories within or accessible to the device. However, as cache memories are generally smaller than the memories further down a memory hierarchy, they cannot hold copies of all of the contents of such lower memories. Thus, performance gains due to improved memory access times are offset by the overhead needed to swap lower memory segments in and out of cache memories.
Generally, cache memory size is fixed, either because the cache memory is a dedicated memory component (e.g., one or more dedicated memory chips or modules), or because a fixed portion of the memory space has been designated as cache memory. Increasing cache memory size can reduce the frequency at which cache load and write back operations occur, as cache miss rates generally decrease with increased cache size. However, once a cache memory is filled with lower memory contents, the cache generally cannot provide any additional memory system performance improvements. If a fixed cache memory size is made very large, the unused cache is not available to other system components, and can be wasted.
Thus, there is a need to more efficiently use cache memory such as not to limit other system components, but allow ample cache when needed.